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WO 99763430 PCT/US99/11534 

SYSTEM AND METHOD FOR COMBINING LOCAL AND 
REMOTE WINDOWS INTO A SINGLE DESKTOP ENVIRONMENT 

Field of the Invention 

The present invention relates to displaying information on remote computers and, in 
particular, to a system and method for combining display data received from various remote 
sources into a single, local display. 

5 Background of the Invention 

Client-server systems, in which a user of a client node is typically remote from a server 
which provides application processing or access to files and other resources, are both convenient 
and cost-effective. Client nodes are generally cheaper than servers, and since one server 
typically provides services to more than one client, overall system cost is reduced. Additionally, 

10 client-server systems allow an enterprise to make decisions regarding the location of certain 

system resources (such as applications) on a situational basis. For example, certain applications 
may be resident solely on clients, solely on servers, solely on certain servers, or any combination 
of the above which improves the overall efficiency of the system. 

To date, however, efforts to combine output data from various sources into a single 

15 display have not met with success. For example, early attempts have been made to cause server- 
based applications to write directly into local windows. Although this method can display 
application output from various servers on a single display, it lacks the ability to arrange the 
windows on the client responsive to the z-axis ordering of the windows at each individual server. 
Thus, if a server brings a new window to the top of its desktop, no corresponding change appears 

20 to the user at the client. 

Further, systems typically cannot support combining various sources of data into a single 
display without modification of the applications generating output data. This results because 
most enterprises desire to use off-the-shelf software to generate output data and such software 
does not support combination of output data. This represents a practical problem because re- 

r 

25 writing such applications to support output combination is generally prohibited by the 
manufacturer of such software and, even if not prohibited, can be expensive. 
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Summary of the Invention 

The present invention relates to a system in which multiple data displays can be 
represented as a cohesive, single, unitary display, without intervention on the part of the user and 
without requiring modification of the applications generating displayed output data. The system 
5 allows a user to interact with displayed windows without knowledge of the source of those 
windows, and changes to the window, either locally or remotely, are reflected in the 
corresponding display on the server or client. 

In one aspect, the present invention relates to a system for incorporating windows from 
remote desktop environments into a local desktop environment. The system includes a local 
10 node, a local agent, a first remote node, and a first remote agent. The first remote node provides 
a first remote desktop environment, and the first remote agent monitors the first remote desktop 
environment for changes in the environment. The first remote node transmits messages to the 
local agent indicative of changes in the first remote desktop environment. The local agent 
receives the transmitted messages and commands the local node to modify a representation of a 
15 first remote window that is part of a local desktop environment. The local agent also monitors 
the local desktop and transmits messages to the remote agent indicative of a change in the local 
desktop. In some embodiments, the local node provides the local desktop environment. 

In another aspect, the present invention relates to a method for incorporating windows 
from remote desktop environments into a local desktop environment. The method comprises the 
20 steps of: providing a local node hosting a local agent; receiving, by the local agent, a message 

indicating a change to windows included in a remote desktop environment; commanding, by the 
local agent, the local node to effect a corresponding change in the local desktop environment; 
monitoring, by the local agent, the local desktop; and transmitting, by the local node, messages 
to the remote node indicative of a change in the local desktop environment. The method may be 
25 embodied on an article of manufacture. 

In yet another aspect, the present invention relates to an agent which incorporates 
windows from remote desktop environments into a local desktop environment. The agent 
includes a message receiving process capable of receiving messages indicating a change has 
occurred in a remote desktop environment. A command process effects changes to the local 
30 desktop environment responsive to messages received by the message receiving process. A 
monitor process monitors local desktop events. A transmission process transmits messages 
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indicating occurrence of the local desktop event. The agent may be embodied on an article of- 
manufacture. 

In a further aspect, the present invention relates to a system for incorporating windows 
from a remote desktop into a local desktop. The system comprises a local node and a remote 
5 node connected by a communications link. The communications link includes a first virtual 

channel and a second virtual channel. The nodes exchange desktop information such as window 
position, window size, and z-ordering of desktop windows, over the first virtual channel. The 
nodes exchange graphical information over the second virtual channel. In some embodiments, 
the first virtual channel and the second channel may be provided as a single virtual channel. 

10 Brief Description of the Drawings 

The invention is pointed out with particularity in the appended claims. The advantages of 
this invention described above, and further advantages, may be better understood by reference to 
the following description taken in conjunction with the accompanying drawings, in which: 
FIG. 1 is a functional block diagram of an embodiment of a client-server system; 
15 FIG. 2 is a functional block diagram of a client node connected to two separate server 

nodes; 

FIG. 3 is a flow diagram of the steps to be taken when a server agent detects a change in 
its associated desktop environment; 

FIG. 4 is a flow diagram of the steps to be taken when a client agent detects a change in 
20 its associated desktop environment; 

FIG. 5 is a flow chart of the steps to be taken to open a virtual channel between a client 
agent and a server agent; and 

FIG. 6 is a functional block diagram of an agent. 

Detailed Description of the Invention 

25 Referring now to Fig. 1, a client-server system is shown in which server node 20 provides 

services for one or more client nodes 10. Client nodes may communicate with server node 20 
via any of a number of industry-standard data communications protocols including, but not 
limited to, TCP/IP, IPX/SPX, NetBEUI, or serial protocols. Alternatively, client nodes 10 may 
connect to server node 20 using a proprietary data communications protocol such as the .ICA 

30 protocol manufactured by Citrix Systems, Inc. of Fort Lauderdale, Florida or the RDP protocol 



OCID: <WO 9960430A1_I_> 



WO 99/63430 



PCT/US99/1 1534 



-4- 

manufactured by Microsoft Corporation of Redmond, Washington. The actual connection 
between the client nodes 10 and the server node 20 may be physical cabling or it may be a 
wireless connection, such as infrared transmission. 

Fig. 2 depicts a system in which a single client node 10 is connected to more than one 
5 server node 20, 20'. As shown in Fig. 2, client node 10 has an associated display 12. The 

display 12 may be used to display one or more components of a graphical user interface, such as 
windows and pull-down menus. The collection of graphical user interface components displayed 
to a user by the display 12 is generally referred to as the "desktop." As shown in Fig. 2, the local 
node 10 displays a local desktop environment 14 to a user. Local node 10 may provide at least a 

10 part of the local desktop environment 14 or local node 10 may simply display various desktop 
components received from other sources such as server nodes 20. As shown in Fig. 2, each 
server node 20, 20' has an associated display 22, 22' which also displays a desktop environment 
24,24'. It should be noted that display 22, 22' need not be a video display monitor. For 
example, display 22, 22' may simply be a bank of video RAM to which applications write the 

15 output of graphical procedure calls. Fig. 2 depicts an embodiment of a system in which each 
server node display 22, 22' displays one graphical user interface window 26, 27'. 

Each server 20, 20' also includes at least one agent 30, 30'. In particular embodiments, 
each server 20, 20' includes one agent 30, 30' for each client 10 connected to the server 20, 20'. 
Client node 10 may also host an agent 40. In some embodiments, a client node 10 hosts a 

20 separate local agent 40 for each server to which the client node 10 is connected. In other 

embodiments, the client node 10 hosts a single agent 40 that manages connections to multiple 
server nodes 20. Each of the agents 30, 30', 40 may monitor their associated desktop 
environment 24, 24', 14 for windows which: change position; are opened; are closed; change 
size; are minimized; are maximized; or are brought to the top of the desktop, i.e., windows which 

25 gain focus that do not previously have focus. Each agent 30, 30', 40 transmits messages 

indicative of changes in their associated desktop 24, 24', 14 to other agents. For example, local 
agent 40 may receive messages transmitted from server node agents 30, 30'. The local agent 40 
commands the client 10 to modify the local desktop environment 14 in response to the messages 
received from server agents 30, 30% that is, the local agent 40 issues commands to the client 

30 node 10 to conform the local desktop environment 14 to the server desktop environment 24. In 
. other embodiments, server node agents 30, 30' receive messages from a local agent 40 and 
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command the server 20, 20' to modify the server desktop environment 24, 24' in response to - 
messages received from the local agent 40. 

In one embodiment, the agents 30, 40 monitor changes to their associated desktop 
environment 24, 24' by periodically issuing one or more of a set of commands provided by the 
5 operating system that allow details of the graphical user interface desktop to be determined. For 
embodiments in which the agents 30, 40 reside on nodes that execute a version of the 
WINDOWS operating system, the agents 30, 40 may periodically issue the EnumWindows 
command to the WINDOWS operating system, which returns a list of all windows present on the 
desktop, together with information related to those windows. The agents 30, 40 can issue the 

10 EnumWindows command every 50 milliseconds, every 100 milliseconds, every 500 

milliseconds, or at any period that allows the agent 30, 40 to rapidly determine when changes to 
its associated desktop environment have occurred without putting a significant computational 
burden on the node. In this embodiment, the agent 30, 40 maintains a data structure storing 
information about the desktop windows and compares the values returned by the EnumWindows 

15 command to the data structure to determine changes. 

Information determined and stored by the agent 30, 30' can include the title bar 
associated with each window, the location of each window in the desktop environment 24, 24 \ 
the size of each window, and the z-order positioning of each window in the desktop environment 
24, 24'. In another embodiment, the agent 30, 30', 40 monitors an intranode graphics message 

20 queue to determine changes to its associated desktop environment. Server agents 30, 30' 

monitor an ihtraserver message queue and local agent 40 monitors an intraclient message queue. 
In this embodiment, changes to the desktop environment 24, 24' are effected via messages sent 
to a graphics subsystem from system applications or the operating system itself. Thus, an 
application executing an a server 20, 20 ? would send a message to a graphics engine residing on 

25 the server 20, 20' in order to change the server desktop environment 24, 24'. Other commands 
which return graphical user interface data are readily apparent to those of ordinary skill in the art. 
For embodiments in which the agents 30, 40 reside on nodes executing a version of the 
WINDOWS operating system, the agents 30, 40 monitor the Windows Message Queue for 
messages affecting the desktop environment associated with the node on which the agent resides. 

30 Examples of such messages include: WMSETFOCUS, which indicates to which window focus 
will be given (i.e., brought to the "top" of the desktop); WM KILLFOCUS, which removes 
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focus from an indicated window; and WM WINDOWPOSC HANGING, which indicates a _ 
change in the position of a window. Other messages that can be posted to the Windows Message 
Queue are readily known to those of ordinary skill in the art. 

Referring now to Fig. 3 ; the steps taken during a server-initiated event are shown. The 

5 server agent 30 senses a change in its associated desktop (step 302). The server agent 30 may do 
this by intercepting a window event on the server message queue, or the agent 30 may determine 
a change in the desktop by comparing the results returned from serially issued operating system 
commands, as described above. The server agent 30 sends a message to a client agent 40 
indicating the change in the server desktop 24 (step 304). For example, if a new window has 

10 been given focus, the server agent 30 can transmit a message to a client agent 40 indicating the 
identity of the new "top" window. In one embodiment, the server agent 30 broadcasts its 
message to all client agents 40 that exist in the system. Alternatively, the server agent 30 may 
transmit its message only to a predetermined subset of client agents 40. For example, when a 
client 10 makes a connection to a server 20, the client agent 40 may register with the server agent 

15 30. In this embodiment, the server agent 30 would transmit change messages only to those client 
agents that have registered with the server. 

The client agent 40 receives the transmitted message (step 306). In embodiments in 
which the server broadcasts commands, the client agent 40 must have some mechanism for 
determining whether a transmitted command affects its associated desktop 12. For example, the 

20 client agent 40 may maintain a list of servers to which it is connected. In these embodiments, the 
client agent 40 responds to messages broadcast by any server present in its list. For 
embodiments in which the server agent 30 does not broadcast messages, no such mechanism is 
necessary. 

The client agent 40 implements a change to its associated desktop 14 responsively to the 
25 received message (step 308). The client agent 40 may accomplish this by directly issuing 

graphics Application Programming Interface commands that cause the client 10 to change the 
display of its associated desktop 14. Alternatively, the client agent 40 may issue GDI commands 
to change its associated desktop 14. In still other embodiments, the client agent 40 issues 
commands directly to the system, whether implemented in hardware or software, responsible for 
30 displaying graphics on the client 10. 
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Referring now to Fig. 4, the steps taken when a client initiates a desktop change are - 
shown. The client agent 40 senses a change in its associated desktop 14 (step 402). As noted 
above, this may be done on an event-driven basis or by polling the operating system operating on 
the client 10. The client agent 40 determines to which server 20 the affected window belongs 
5 (step 404). To facilitate this process, the client agent 40 may maintain a list that associates 
remote windows with a particular server 20. The client agent 40 then sends a message to the 
identified server 40 indicating the change in its desktop 14 (step 406). Alternatively, the client 
agent 40 may skip step 404 entirely and broadcast its change message to all servers 20. The 
server agent receives the transmitted message (step 408) and implements the change in its 

10 associated desktop (step 410), as described above. 

In one particular embodiment, a client node 10 and a server node 20 communicate using 
the ICA protocol and the client node 10 and the server node 20 execute a version of the 
WINDOWS operating system. Client node 10 hosts a local agent 40 that may be provided as a 
dynamically linked library module. The server node 20 hosts a server agent 30 that may be 

15 provided as a separate thread. 

In this embodiment, the local agent 40 and the server agent 30 exchange graphical data, 
i.e., the data actually displayed in each window on the desktop, via a first ICA virtual channel. 
Information about window positioning, window size, z-access ordering of window and other 
such information is communicated between the client node 10 and the server node 20 via a 

20 second ICA virtual channel. Throughout the description, when the client node 10 and the server 
node 20 are actively exchanging information via the second ICA virtual channel, the client will 
be referred to as being in "seamless windowing mode." 

Referring now to Fig. 5, the process for enabling seamless windowing mode between the 
local agent 40 and server agent 30 is shown. In this embodiment, all communication between a 

25 server agent and a client agent is packet-oriented and takes place over a dedicated ICA virtual 
channel, making the functioning of the agents 30, 40 independent from the underlying 
communication protocol. All packets start with packet type (1 byte), followed by packet data 
length (2 bytes, can be zero) and data (optional). Agents 30, 40 will try to send as much data in a 
single network packet as possible, but it will always send complete packets. That is, the size of 

30 seamless window virtual packets never exceeds the allowable size of an ICA packet. Packet flow 
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control and delivery confirmation is implemented by the transport level of the ICA protocol. - 
Individual packets are executed immediately on reception. 

The client agent 40 waits for an initial packet from the server agent 30. After user logon 
to the server, a server agent 30 will be invoked (step 504). 

5 The server agent sends a TWI_PACKET_START packet to the client agent 40, which 

includes some essential information about the server desktop environment (desktop resolution, 
desktop size, version number of ICA protocol supported by the server, etc.) (step 506). This 
packet is sent by the server agent 30 on initial connection or on reconnect, and is used to: (1) 
detect seamless windowing capabilities of the client; and (2) requests basic client information. 

10 The client agent receives the TWI_PACKET_START packet (step 507) and responds 

with a TWI_PACKET_C2H_START_ACK packet, confirming TWI_PACKET_START and 
supplying client version/capabilities information (step 508). This packet is sent by the client 
agent 40 to confirm reception of TWI_PACKET_START packet and to send the requested basic 
client information to the server agent 30. 

15 If there is no response from the client agent 40 (step 509), the server agent 30 assumes 

that the client is unable to enter seamless windowing mode, and the seamless windowing virtual 
channel is not used by the server node 20 to communicate window information. In this case, the 
server node 20 continues to communicate graphical data to the client node 10 via another virtual 
channel, and the client desktop displays the server desktop without incorporating windows from 

20 other nodes. 

The client agent 40 uses the information sent by the server agent 30 in step 506 to 
determine if a seamless windowing session can be established between the server agent 30 and 
the client agent 40. In one embodiment, the client agent 40 compares information relating to the 
version of the virtual channel protocol supported by the server agent 30 to makes the 
25 determination If the client agent 40 determines that it is possible to enable seamless windowing 
mode (step 510), the client agent 40 sends a TWI_PACKET_C2H_OPEN packet to the server 
agent 30 (step 511). This packet requests that the server agent 30 enable seamless windowing 
mode. 

On reception of a TWI_PACKET_C2H_OPEN packet (step 512) the server agent 40 (I) 
30 resets its internal data structures, (ii) sends a TWI_PACKET_SYSINFO packet to the client 

agent 40 to communicate some general information regarding the window settings on the server 
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node 20 to the client agent' 40, (iii) sends a TWI_PACKET_OPEN packet to the client agent 40 
(step 514) indicating the establishment of seamless windowing mode, and (iv) enables its main 
polling loop (step 516) that will poll the operating system on the server node for desktop 
changes. If the client agent 40 and the server agent 30 do not support the same version of the 
5 seamless window protocol, the server, agent 30 ignores the TWI_PACKET_C2H_OPEN packet. 

On reception of TWIPACKETJ3PEN packet (step 520), the client agent 40 resets its 
internal data structures (step 522) and seamless windowing mode between the client agent 40 and 
the server agent 30 is established. 

During a seamless windowing mode session, the server agent 30 will send window 

10 information such as window position, size, styles, window text, etc. for all top-level windows on 
the server node. Also, foreground window information is sent, i.e., which window on the server 
node desktop is. the foreground window. In accordance with this information, the client agent 40 
creates windows with the same size/position as the server node windows on the client node 
desktop. In some embodiments, window elements are transmitted as bitmaps from the server 

15 node 20. Examples of packets sent by the server agent 30 include : TWIPACKETCLOSE, 

which is sent to switch the client agent 40 out of seamless windowing mode and back to regular, 
or full screen, mode; that is, the client node 10 is switched back to displaying the server node 
desktop environment without incorporating windows from other desktop environments; 
TWI_PACKET_CREATEW, which is sent to create new windows on the client node 10; 

20 TWI_PACKET_DELETEW, which is sent to destroy a window on the client node 10; 

TWI_PACKET_CHANGEW, which is sent to change a window displayed by the local node 10; 
TWI_PACKET_SYSINFO, which is sent to report server node 20 system settings — normally it 
is sent only once, but the packet can be sent multiple times; TWI_PACKET_FOREGROUNDW, 
which is sent during normal seamless windowing mode operation to change the foreground 

25 window; TWI_PACKET_SETTOPW, which is sent during normal seamless windowing mode 
operation to change the top window, that is, to bring a new window to top; 

TWI_PACKET_SETFOCUS, which is sent during normal seamless windowing mode operation 
to change the focus window; TWI_PACKET_FOCUSACK 5 which is sent in response to 
TWI_PACKET_C2H_SETFOCUS (see below), and reports the result of a SetFocus attempt; and 
30 TWI_PACKET_SPA_STATUS, which is sent in response to 
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TWI_PACKET_C2H_START_PUBLICAPP (see below), and is used to report the result of the 
requested operation. 

Examples of packets that can be sent by the client agent 40 to the server agent 30 include 
: TWI_PACKET_C2H_PAUSE, which is sent to suspend the server agent 30, that is, the server 
5 agent 30 will stop sending window information, clear its internal data structure and send a 

TWI _PACKET_CLOSE packet (see above); TWI_PACKET_C2H_RESUME, which is sent to 
resume the server agent 30 - the server agent 30 will clear its internal data structure, and send a 
TWI J>ACKET_OPEN packet (see above); TWI_PACKET_C2H_SETPOS, which is sent to 
report window size/position change on the client node; TWI_PACKET_C2H_SETFOCUS> 

10 which is sent to report a change in the focus window on the client node; 

TWI_PACKET_C2H_RESTORE, which is sent to request restoration of a minimized window; 
TWI_PACKET_C2H_TERMINATE, which is sent to request termination of a program 
executing on the server node 20; TWI_PACKET_C2H_STARTAPP, which is sent to start a new 
application on the server node 20; TWI_PACKET_C2H_LOGOUT, which is sent to end the 

15 current session; TWI_PACKET_C2H_START_PUBLICAPP, which is sent to start a new 

published application on the server node; and TWI_PACKET_C2H_CLIENTINFQ, which is 
sent to report client desktop settings to the server agent 30 — this packet is generally sent on 
startup, but can also be used during seamless windowing session. 

The client agent 40 will try to perform some operations (such as window move and 

20 resize) locally, sending update information back to the server node 40 afterwards. ^Proper 

window behavior is emulated by intercepting the WM_NCHITTEST message for the client- 
created windows. 

Foreground window changes can happen on both the client node and the server node, so 
the client and server will negotiate and balance actual foreground window changes. For 

25 example, if the server node 20 changes its foreground window, that change should be properly 
represented on the client desktop. The server agent 30 sends information regarding the new 
foreground window to the client agent 40 using the TWI_PACKET_FOREGROUNDW packet. 
Similarly, if the client agent 40 detects a foreground window change on the client desktop, the 
client agent 40 sends information regarding the change to the server agent 30 and the server 

30 agent 30 implements the change on the server desktop. 
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When focus is taken away from a window representing a server window and is given to a 
local client window, the client notifies the server of the change and the server gives focus to an 
invisible window. For embodiments in which the client node 10 is connected to two server 
nodes 20, and focus is shifted from a window representing a window from the first server and is 
5 given to a window representing a window from the second server, the client sends a packet 
informing the current server that its window no longer has focus. Once the server responds by 
giving focus to an invisible window, the client agent 40 instructs the other server that its window 
now has focus on the local desktop. 

In some embodiments, it is desirable to add some complexity to the agent's main polling 

10 loop to reduce network traffic. In these embodiments, the main polling loop includes a 
comparison between the current foreground window and the identity of the window last 
requested to be moved to the foreground. If the current foreground window matches the window 
identified in the most recent request, the agent does not need to send information acknowledging 
the change. This technique is useful in both server agent 30 and client agents 40. 

15 Window z-ordering on the client is a superset of the server node z-ordering (client will 

always have more windows than the host). Server node z-ordering is reproduced on the client by 
reproducing owner/owned relationship among windows and the TOP_MOST flag in the window 
style. Owner/owned relationships refer to windows which are children of other windows, such as 
dialog boxes associated with application windows. The dialog box is said to be owned by the 

20 application window, and the dialog box will always appear on top of its owner. The TOP_MOST 
flag indicates that a particular window should appear on "top" of the desktop, for example, the 
status bar in WINDOWS 95. 

When a user disconnects, the server agent 30 switches itself to suspended mode, and will 
not send information to the client agent 40. On a reconnect, the server agent 30 sends a 

25 TWI_PACKET_START packet, reporting HostAgentState as "already running, reconnect." 

Based on the version number of the protocol supported by the server the client will decide 
whether it is possible to enable seamless windowing mode (from the client point of view). If it is 
possible to switch to seamless windowing mode, the client agent 40 will send a 
TWI_PACKET_C2H_OPEN packet, asking the server agent 30 to enable seamless windowing 

30 mode. 
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Each agent responsible for monitoring an associated desktop may be implemented as a 
stand-alone software routine (such as an executable file on DOS-based systems), a dynamically 
linked library routine (DLL), or as an integral piece of the operating system. Referring now to 
FIG. 6, and in brief overview, each agent includes a message receiving facility 602, a command 

5 facility 604, a monitor facility 606, and a message transmission facility 608. Agent-agent 
communication is full-duplex, i.e., agents can transmit and receive messages simultaneously. 
Thus, each facility can be implemented as a separately functioning code segment that operates 
independently of the other facilities. For example, message receiving facility 602 and command 
facility 604 can be implemented as separate threads which communicate with each other via a 

10 named pipe or shared memory. Use of a common data allows the message receiving facility 602 
and the message transmitting facility 608 to be synchronized . 

Message receiving facility 602 receives messages transmitted from other agents 
indicating changes in the desktop environments associated with those agents. Message receiving 
facility 602 may connect directly with the physical layer of the communications protocol the 

15 - agents use to communicate, or the message receiving facility 602 may operate at a higher layer of 
the protocol by cooperating with one or more communications subsystems. For embodiments in 
which messages are broadcast by agents, the message receiving facility 602 has some mechanism 
for determining whether a broadcast message is intended for it. For example, the message 
receiving facility 602 may store a list of the windows which its associated desktop displays. The 

20 message receiving facility 602 would compare the target of any received message to its list of 
windows to determine whether or not to take action on the received message. The message 
receiving facility may be implemented as a blocking function. Alternatively, the message 
receiving facility can be implemented a call-back function invoked by the ICA virtual channel 
transport. 

25 Once the message receiving facility 602 has determined that a received message is 

intended for its desktop, the command facility is invoked to effect the change indicated by the 
message to the associated desktop environment. The command facility 604 may be passed the 
received message facility, or the message receiving facility 602 may process the received 
message before communicating with the command facility 604. The command facility 604 may 

30 implement the desktop change indicated by the received message by issuing GDI commands. In 
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other embodiments, the command facility 604 may issue commands directly to an associated - 
graphics subsystem or may issue other graphics API commands. 

During a seamless windowing session, a number of desktops* are associated with a single 
client node - one desktop on the client itself and one desktop per server node 20 to which the 
5 client node 10 is connected. The client agent 40, in conjunction with the server agent 30, 30\ 
creates a combined window list representing the z-order of all desktops. All participating 
desktops are "linked" together by the client agents 40 and the server agents 30, 30', and any z- 
order changes on any desktops will be propagated to other desktops. 

In one embodiment, each server has knowledge only of its own graphical desktop 

10 representation and the server desktops are individually represented within the client. The client 
display is updated by combining all server and client desktop images into a single display image 
based on the window information that has been obtained from each server node 20, 20' by the 
client agent 40. The resulting image is displayed at the client node 10. 

The combining process involves building a common window list based on the windows 

15 information exchanged by all agents. Using the combined window list, the graphical desktop 
data is clipped and merged for representation by the client node 10. The node takes care of 
"clipping" displayed windows resulting from the commands issued by the command facility 604. 
Such "clipping'' functions are well-known to those of ordinary skill in the art. In some 
embodiments, however, the command facility 604 maintains a shadow bitmap of clipped 

20 windows. That is, the command facility 604 maintains a bit image of windows that are obscured 
by other windows. This allows the agent to change its associated desktop without requiring it to 
reload the window image of an obscured window from the appropriate source. In other 
embodiments, the node determines whether graphical data is obscured at the time it is received. 
If it is, the node ignores the received graphical data. If it is not, the node displays the data. The 

25 node makes a determination as to whether the graphical data is obscured by applying clipping 
functions. 

Monitoring facility 606 monitors the desktop associated with the agent. Monitoring 
facility 606 may monitor the desktop by periodically issuing commands provided by the 
operating system executing on the node which return information about the node's desktop. 
30 Alternatively, the monitoring facility 506 may watch for messages posted to an intranode 
message queue. As noted above, in one particular embodiment the monitoring facility 606 



OCID: <WO 9963 430 A 1 J _> 



WO 99/63430 



PCT/US99/11534 



-14- 

monitors the Windows Message Queue. Once a desktop change occurred, the message 
transmission facility 608 transmits a message indicating the change that has occurred. In some 
embodiments, the message transmission facility 608 broadcasts notification of the change. 

In one embodiment, message transmission facility 608 can be implemented in the form of 
non-blocking function, that can be called from any window procedure. If the function can not 
send a data packet immediately (for example, the communication subsystem has no buffer 
space), a timer will be set and retry attempts will be done until the send succeeds. 

The present invention may be provided as one or more computer- readable programs 
embodied on or in one or more articles of manufacture. The article of manufacture may be a 
floppy disk, a hard disk, a CD ROM, a flash memory card, a PROM, a RAM, a ROM, or a 
magnetic tape. In general, the computer-readable programs may be implemented in any 
programming language. Some examples of languages that can be used include C, C-J-+, or 
JAVA. The software programs may be stored on or in one or more articles of manufacture as 
object code. 

Having described certain embodiments of the invention, it will now become apparent to 
one of skill in the art that other embodiments incorporating the concepts of the invention may be 
used. Therefore, the invention should not be limited to certain embodiments, but rather should 
be limited only by the spirit and scope of the following claims. 
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What is claimed is: 



1 1. A system for incorporating windows from remote desktop environments into a local 

2 desktop environment, the system comprising: 

3 a local node having a local desktop environment; 

4 a local agent; 

5 a first remote node having a first remote desktop environment including at least one 

6 first remote window; and 

7 a first remote agent monitoring said first remote desktop environment and in 

8 communication with said local agent, said first remote agent transmitting a message to 

9 said local agent indicative of a change to said first remote window, 

10 said local agent receiving the message transmitted by said first remote agent and, 

1 1 responsive to the received message, commanding said local node to modify a 

12 representation of said first remote window as part of said local desktop environment and 

13 said local agent transmitting messages to said first remote agent indicative of a locally- 

14 generated change to a representation of said first remote window. 

1 2. The system of claim 1 wherein said local node provides said local desktop environment, 

2 said local agent monitors said local desktop environment, and wherein said local agent 

3 receives the messages transmitted by said remote agent and, responsive to the received 
4. message, commands said local node to modify a representation of said first remote 

5 window as part of said local desktop environment. 

1 3. The system of claim 1 wherein said first remote agent transmits a message indicating that 

2 said first remote window has changed position and wherein said local agent receives the 

3 message transmitted by said first remote agent and commands said local node to change 

4 the position of a representation of said first remote window in said local desktop 

5 environment. 

1 4. The system of claim 1 wherein said first remote agent transmits a message indicating that 

2 said first remote window has closed and wherein said local agent receives the message 

3 transmitted by said first remote agent and commands said local node to close a 

4 representation of said first remote window in said local desktop environment. 
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1 5. The system of claim 1 wherein said first remote agent transmits a message indicating that 

2 said first remote window has changed size and wherein said local agent receives the 

3 transmitted message and commands said local node to change the size of a representation 

4 of said first remote window in said local desktop environment. 

1 6. The system of claim 1 wherein said first remote agent monitors said first remote desktop 

2 environment by monitoring an intranode message queue. 

1 7. The system of claim 1 wherein said local node provides said local desktop environment 

2 and said local agent monitors said local desktop environment. 

1 8. The system of claim 7 wherein said local node broadcasts a message indicative of a 

2 locally-generated change to a representation of said first remote window. 

1 9. The system of claim 1 wherein said remote node broadcasts a message indicative of a 

2 change to said first remote window. 

1 10. The system of claim 1 wherein said local node performs clipping functions on said 

2 representations of said first remote window that are obscured by local windows or other 

3 remote windows. 

1 11. A method for incorporating windows from remote desktop environments into a local 

2 desktop environment, the method comprising the steps of: 

3 (a) providing a local node hosting a local agent; 

4 (b) receiving, by the local agent, a message indicating a change to windows 

5 included in a remote desktop environment; 

6 (c) commanding, by the local agent, the local node to effect a corresponding 

7 change in the local desktop environment; 

8 (d) monitoring, by the local agent, the local desktop environment; and 

9 (e) transmitting, by the local agent, a message indicative of a change to 
10 representation of the remote windows included in the local desktop environment. 

1 12. The method of claim 1 1 wherein steps (b) and (c) comprise: 

2 (b) receiving, by the local agent, a message indicating that a window included in a 
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3 remote desktop environment has changed position; and 

4 (c) commanding, by the local agent, the local node to change the position of a 

5 representation of the remote window in the local desktop environment. 

1 13. The method of claim 1 1 wherein steps (b) and (c) comprise: 

2 (b) receiving, by the local agent, a message indicating that a window included in a 

3 remote desktop environment has closed; and 

4 (c) commanding, by the local agent, the local node to close a representation of the 

5 remote window in the local desktop environment. 

1 14. The method of claim 1 1 wherein steps (b) and (c) comprise: 

2 (b) receiving, by the local agent, a message indicating that a window included in a 

3 remote desktop environment has changed size; and 

4 (c) commanding, by the local agent, the local node to change the size of a 

5 representation of the remote window in the local desktop environment. 

1 15. An agent for incorporating windows from remote desktop environments into a local 

2 desktop environment, the agent comprising: 

3 a message receiving process capable of receiving messages indicative of a change in a 

4 remote desktop environment; 

5 a command process capable of effecting changes to a local desktop environment 

6 responsive to the received messages; 

7 a monitor process capable of monitoring local desktop events; and 

8 a transmission process capable of transmitting messages indicative of the local 

9 desktop events. 

1 16. The agent of claim 1 5 wherein said transmission process broadcasts messages indicative 

2 of the local desktop events. 

1 17. The agent of claim 15 wherein said command process issues application programming 

2 interface commands, responsively to received messages, to effect changes to a local 

3 desktop environment. 

1 18. A article of manufacture having the agent of claim 15 embodied thereon. 
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1 19. A system for incorporating windows from remote desktop environments into a local - 

2 desktop environment, each desktop environment including at least one window displaying 

3 graphical data, the system comprising: 

4 a communications link comprising a first virtual channel and a second virtual 

5 channel; 

6 a local node; and 

7 a remote node, 

8 said local node and said remote node exchanging window information over said first 

9 virtual channel of said communications link and said local node and said remote node 

10 exchanging graphical data over said second virtual channel of said communications link. 

1 20. The system of claim 20 wherein the first virtual channel and the second virtual channel 

2 comprise the same virtual channel. 
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